Videos

The Intelligent Trial and Error Algorithm introduced in the paper 'Robots that can adapt like animals' (Nature, 2015): the video shows two different robots that can adapt to a wide variety of injuries in under two minutes. A six-legged robot adapts to keep walking even if two of its legs are broken, and a robotic arm learns how to correctly place an object even with several broken motors.

Full citation: Cully A, Clune J, Tarapore DT, Mouret J-B. Robots that can adapt like animals. Nature, 2015. 521.7553, (cover article).

Enabling artificial agents to automatically learn complex, versatile and high-performing behaviors is a long-lasting challenge. This paper presents a step in this direction with hierarchical behavioral repertoires that stack several behavioral repertoires to generate sophisticated behaviors. Each repertoire of this architecture uses the lower repertoires to create complex behaviors as sequences of simpler ones, while only the lowest repertoire directly controls the agent's movements. This paper also introduces a novel approach to automatically define behavioral descriptors thanks to an unsupervised neural network that organizes the produced high-level behaviors. The experiments show that the proposed architecture enables a robot to learn how to draw digits in an unsupervised manner after having learned to draw lines and arcs. Compared to traditional behavioral repertoires, the proposed architecture reduces the dimensionality of the optimization problems by orders of magnitude and provides behaviors with a twice better fitness. More importantly, it enables the transfer of knowledge between robots: a hierarchical repertoire evolved for a robotic arm to draw digits can be transferred to a humanoid robot by simply changing the lowest layer of the hierarchy. This enables the humanoid to draw digits although it has never been trained for this task.

Supplementary video of: Antoine Cully and Yiannis Demiris. 2018. Hierarchical Behavioral Repertoires with Unsupervised Descriptors. In Proceedings of Genetic and Evolutionary Computation Conference, Kyoto, Japan, July 15–19, 2018 (GECCO ’18), 8 pages. DOI: 10.1145/3205455.3205571

Numerous algorithms have been proposed to allow legged robots to learn to walk. However, their vast majority are devised to learn to walk along a straight line, which not sufficient to accomplish any real-world mission. Here we introduce TBR-Learning, a new learning algorithm that simultaneously discovers several hundreds of simple walking controllers, one for each possible direction. By taking advantage of solutions that are usually discarded, TBR-Learning is substantially faster than independently learning each controller. Our technique relies on two methods: (1) novelty search with local competition, which comes from the artificial life research field and (2) the transferability approach, which combines simulations and real tests to optimize a policy. We evaluate this new technique on a hexapod robot. Results show that with only a few dozens of short experiments performed on the physical robot, the algorithm learns a collection of controllers that allows the robot to reach each point of its reachable space. Overall, TBR-Learning opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.

Learning in robotics typically involves choosing a simple goal (e.g. walking) and assessing the performance of each controller with regard to this task (e.g. walking speed). However, learning advanced, input-driven controllers (e.g. walking in each direction) requires testing each controller on a large sample of the possible input signals. This costly process makes difficult to learn useful low-level controllers in robotics. Here we introduce BR-Evolution, a new evolutionary learning technique that generates a behavioral repertoire by taking advantage of the candidate solutions that are usually discarded. Instead of evolving a single, general controller, BR-evolution thus evolves a collection of simple controllers, one for each variant of the target behavior; to distinguish similar controllers, it uses a performance objective that allows it to produce a collection of diverse but high-performing behaviors. We evaluated this new technique by evolving gait controllers for a simulated hexapod robot. Results show that a single run of the EA quickly finds a collection of controllers that allows the robot to reach each point of the reachable space. Overall, BR-Evolution opens a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.